Projector with spatial light modulation

ABSTRACT

A time of flight based depth detection system is disclosed that includes a projector configured to sequentially emit multiple complementary illumination patterns. A sensor of the depth detection system is configured to capture the light from the illumination patterns reflecting off objects within the sensor&#39;s field of view. The data captured by the sensor can be used to filter out erroneous readings caused by light reflecting off multiple surfaces prior to returning to the sensor.

BACKGROUND OF THE INVENTION

Numerous techniques exist for range imaging, which can be quite usefulin multiple different applications. One specific type of range imagingcan be performed using a time of flight camera. The time of flightcamera can measure the time it takes for a pulse of light to travel toand from objects in the sensor's field of view to determine the distancebetween the sensor and the objects in the sensor's field of view.Unfortunately, light emitted by a depth detection system may not alwaystravel directly to an object within the sensor field of view and back tothe sensor. If the light bounces off another object before reflectingoff the object, the time taken for the light to return to the sensor isincreased, thereby increasing the measured time of flight for areflected pulse of light. The longer time of flight measurement canresult in the depth detection system erroneously increasing the measureddistance between the sensor and the object. Consequently, a way offixing this error is desirable.

SUMMARY OF THE INVENTION

This disclosure describes a time of flight camera configured to filterout erroneous readings resulting from pulses of light bouncing offmultiple surfaces.

This disclosure relates to ways of improving performance of a depthdetection system. The depth detection system can be configured tosequentially emit complementary illumination patterns onto a regionbeing monitored by an imaging sensor of the depth detection system. Theimaging sensor can act as a time of flight sensor to determine adistance between the depth detection system and objects within theregion by measuring the time it takes for light forming the illuminationpatterns to reflect off the objects and return to the imaging sensor.Some of the light received at the imaging sensor can be indirect lightthat bounces off other surfaces before arriving at the imaging sensor.This can be especially problematic in room corners where more indirectlight is likely to return to the imaging sensor. The reflectionsincrease the amount of time it takes for the light to return to theimaging sensor, thereby reducing the accuracy of the sensor data. Someof this indirect light can be filtered out from consideration by thedepth detection system by identifying light reflecting off portions ofthe region being monitored by the imaging sensor falling outside of afirst illumination pattern when the first illumination pattern isactive. This identified light can then be subtracted out ofconsideration when the second illumination pattern is active. Similarly,light falling outside of the second illumination pattern when the secondillumination pattern is active can be subtracted from the firstillumination pattern. In this way, more accurate depth detectioninformation can be obtained.

Light sources that emit the complementary illumination patterns can bemounted to a common substrate to prevent the light sources from beingout of alignment from one another. The common substrate can also helpreduce any thermal effects that would result in the light sources beingthrown out of alignment.

A depth detection system is disclosed and includes at least thefollowing: a projection system, comprising: a projector housing having arigid substrate, a first light source configured to emit light through afirst plurality of light shaping components, the first light sourcebeing mounted to the rigid substrate, and a second light sourceconfigured to emit light through a second plurality of light shapingcomponents, the second light source being mounted to the rigid substrateadjacent to the first light source; an imaging sensor proximate theprojection system and configured to receive light emitted by the firstand second light sources after being reflected off objects within afield of view of the imaging sensor; and a processor configured tocalculate a distance between the depth detection system and the objectswithin the sensor field of view by measuring an amount of time for lightemitted by the first and second light sources to reflect off the objectswithin the sensor field of view and return to the imaging sensor.

Another depth detection system is disclosed and includes the following:a plurality of light shaping components, comprising: a collimatingoptical element, a refractive optical element, a diffractive opticalelement, and a micro-lens array; a light source configured to emit lightthrough the plurality of light shaping components; an imaging sensorconfigured to detect light emitted by the light source and reflected offobjects within a field of view of the imaging sensor; and a processorconfigured to determine a distance between the depth detection systemand the objects by filtering out sensor readings associated with lightreflected off surfaces outside the field of view of the imaging sensor.

A depth detection system is disclosed and includes the following: aprojection system, comprising: a projector housing having a rigidsubstrate, a first light source configured to emit light through a firstplurality of light shaping components and produce a first illuminationpattern, the first light source being mounted to the rigid substrate,and a second light source configured to emit light through a secondplurality of light shaping components and produce a second illuminationpattern complementary to the first illumination pattern, the secondlight source being mounted to the rigid substrate adjacent to the firstlight source; an imaging sensor proximate the projection system andconfigured to receive light emitted by the first and second lightsources after being reflected off objects within a field of view of theimaging sensor; and a processor configured to calculate a distancebetween the depth detection system and the objects within the sensorfield of view by measuring an amount of time for light emitted by thefirst and second light sources to reflect off the objects within thesensor field of view and return to the imaging sensor and filtering outsensor readings associated with light reflected off surfaces outside thefield of view of the imaging sensor.

Other aspects and advantages of the invention will become apparent fromthe following detailed description taken in conjunction with theaccompanying drawings which illustrate, by way of example, theprinciples of the described embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will be readily understood by the following detaileddescription in conjunction with the accompanying drawings, wherein likereference numerals designate like structural elements, and in which:

FIG. 1A shows an exemplary depth detection sensor in use;

FIG. 1B shows how light incident to an object can be reflected bydiffuse and/or specular reflection according to some embodiments;

FIG. 1C shows examples of different types of objects illuminated by aprojection system according to some embodiments;

FIG. 2A shows a projection system 102, which includes two projectorsaccording to some embodiments;

FIG. 2B shows exemplary illumination patterns A and B according to someembodiments;

FIG. 2C shows illumination patterns C and D according to someembodiments;

FIG. 2D shows illumination patterns E, F and G according to someembodiments;

FIG. 2E shows how discrete pixels or sampling points can be distributedacross multiple illumination patterns according to some embodiments;

FIGS. 3A-3C show various optics assembly embodiments, which are eachmade up of a group of light shaping components positioned in front of alight source according to some embodiments;

FIGS. 4A-4B show a projector assembly with two light sources thatincorporates an optics assemblies for each light source similar to theoptics assembly depicted in FIG. 3 according to some embodiments;

FIGS. 5A-5C show views of a multiple light source projector assemblyutilizing folded optics according to some embodiments;

FIGS. 6A-6B show side views of a projection assembly using a singlelight source according to some embodiments; and

FIG. 7 shows a diagram depicting interaction between differentcomponents of the aforementioned depth detection system according tosome embodiments.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Representative applications of methods and apparatus according to thepresent application are described in this section. These examples arebeing provided solely to add context and aid in the understanding of thedescribed embodiments. It will thus be apparent to one skilled in theart that the described embodiments may be practiced without some or allof these specific details. In other instances, well known process stepshave not been described in detail in order to avoid unnecessarilyobscuring the described embodiments. Other applications are possible,such that the following examples should not be taken as limiting.

A depth detection system can be configured to characterize anenvironment within a field of view of the depth detection system. Theresulting characterization can be used to determine a position andexterior shape of portions of objects facing the depth detection system.One type of depth detection system is a time of flight (TOF) camera. ATOF camera utilizes a projector for emitting modulated pulses of lightand a sensor for receiving a portion of each of the pulses of light thatreflects off the various objects within the sensor's field of view. Aprocessor receiving readings from the sensor can determine the timetaken for the light to travel from the sensor and bounce off one of theobjects in the field of view and return to the sensor. Because the speedof light is known, the system can determine the distance between thedepth detection sensor and the object based on that time. Unfortunately,while this method works well for determining distance when the lightbounces off an object and returns directly back to the sensor, any lightreturning to the sensor that bounces off another object first can causeinaccuracies in the depth data.

One solution to this problem is to filter out indirectly reflected lightreceived at the sensor to reduce inaccuracies. One way this can beaccomplished is to adjust the manner in which the environment isilluminated with light. The light can be emitted by a projection systemin alternating illumination patterns to sequentially illuminatedifferent portions of the objects in the field of view. In someembodiments, the illumination pattern can be arranged in substantiallyparallel stripes, although different patterns are also possible. Each ofthe stripes can be separated by a gap having about the same thickness aseach stripe. In this way, about half of the field of view can beilluminated any time an illumination pattern is emitted. It should beappreciated that different stripe and gap thicknesses can be used butthat at some point during a series of different illumination patternseach portion of the field of view should be unilluminated. Any lightreturning from areas of the frame that should not be illuminated by aparticular pattern of light can be used to identify reflected light.When a different illumination pattern illuminates that portion of theobject from which reflected light was previously detected, the reflectedlight can be subtracted from the detected light to identify only thatportion of the light that travels directly from the projection system tothe object and back to the sensor. Any other light can be ignored forthe purposes of making a depth map of the area with the sensor's fieldof view. In this way, the accuracy of the depth data can besubstantially improved.

A projection system for performing the aforementioned method can includetwo or more light sources for generating the illumination patterns. Insome embodiments, the projection system can be configured to operatevery quickly in order to keep up with changing conditions. For example,in some embodiments, the light sources can be configured to emit morethan 100 pulses per second. A sensor associated with the projectionsystem can be configured to capture the light as it comes back and canhave a global shutter that allows each of the pixels of the sensor to beread at the same time. In this way, any errors introduced due tosequentially reading the pixels can be avoided.

In some embodiments, the light sources can be incorporated within asingle projector housing. Packaging the light sources in a singleprojector prevents the situation where one of two or more separateprojection units gets bumped or jostled a different amount than theother units, which results in misalignment of the illumination patterns.While a slight change in alignment of a single projector configured toproject multiple illumination patterns could result in a portion of thesensor field of view not being covered by the illumination pattern, themajority of the sensor field of view could remain covered withoutcompromising alignment of the illumination patterns. In someembodiments, a single projector housing can include a unitary rigidsubstrate with a low coefficient of thermal expansion that keeps theseparation between the light sources consistent over a large range oftemperatures. Each of the light sources can have different optics thatdirect the light into the various illumination patterns. In someembodiments, a projection system with a single light source can be usedthat has shifting optics. In such an embodiment, the optics canoscillate between two or more positions to create two or moreillumination patterns from the single light source.

These and other embodiments are discussed below with reference to FIGS.1A-7; however, those skilled in the art will readily appreciate that thedetailed description given herein with respect to these figures is forexplanatory purposes only and should not be construed as limiting.

FIG. 1A shows an exemplary depth detection system 100 in use. Depthdetection system 100 includes a projection system 102 and a sensor 104.Projection system 102 can be configured to emit light towards an object106. In some embodiments, the light emitted by projection system 102 canbe infrared light or near infrared light. Since the light emitted byprojection system 102 can be configured to cover a broad areacorresponding to a field of view of sensor 104, exemplary light wave 108can bounce off of wall 110 and due to the angle of wall 110 light wave108 can instead of reflecting back from wall 110 bounce off object 106and then back to sensor 104 as depicted. This can be particularlyproblematic when object 106 has irregular surfaces (i.e. curved orcylindrical surfaces) that scatter light incident to object 106. Thescattering of the reflected light increases the likelihood of thereflected light arriving back at sensor 104 as depicted.

FIG. 1B shows how light incident to object 106 can be reflected bydiffuse and/or specular reflection. While a flat surface is generallyneeded to generate specular reflection, a flat surface also tends togenerate some diffuse reflection on account of scattering centerslocated below the surface of object 106. Curved or varied surfacesgenerate even more diffuse reflections that scatter in many directions.One of the reasons the light reflected off wall 110 can be hard todistinguish from the direct light is that when the surface of wall 110is relatively flat, a substantial amount of light wave 108 can bereflected as specular reflection from wall 110, thereby causing theresulting diffuse reflection at object 106 from light wave 108 to have asimilar intensity as the diffuse reflection at object 106 originatingfrom light wave 112. It should be noted that light going from theprojector to object 106 and then bouncing off wall 110 back towards thesensor is not considered to be a problem where wall 110 is not in thesensor field of view. In such a case, the high angle of incidence of thelight entering the sensor would not be detected by the sensor on accountof the sensor only being configured to receive light arriving from aparticular field of view. The high angle of incidence light can beprevented from reaching the sensor using a shroud or light gatheringlens positioned over the sensor.

FIG. 1C shows examples of different types of objects illuminated byprojection system 102. The first column of images shows images generatedusing all of the light reflected off of the objects and captured bysensor 104. The images in the second column show only the lightreflected directly off the objects. The images in the third column showonly the light reflected first off other objects (indirect light) priorto hitting the object in the sensor field of view. The first row of eggpictures provide an example of diffuse interreflections. The sphericalshape of the eggs accentuates the amount of diffuse reflection generatedby light striking the surface of each of the eggs. In particular, theindirect light image from the first row shows how the lower edges of theeggs capture a substantial amount of indirect light and couldconsequently appear to be farther away from the sensor. The second rowof wooden block pictures provides an example of both diffuse andspecular interreflection. The flat surfaces of the wooden blocks allowfor a certain amount of specular reflection while the underlying woodgrain structure and corners of the blocks create diffuseinterreflection. Finally the third row of peppers shows how sub-surfacescattering can cause only a small amount of light to be reflecteddirectly back to sensor 104. This limited amount of direct light canmake filtering out the indirect light even more important fordetermining the actual distance between sensor 104 and the peppers. FIG.1C was originally published as part of the article “Fast Separation ofDirect and Global Components of a Scene using High FrequencyIllumination”, by Krishnan.

FIG. 2A shows projection system 102, which includes projectors 202 and204. Projectors 202 and 204 can be used to emit complementaryillumination patterns A & B. Illumination patterns A & B can besequentially pulsed so that only one of the illumination patterns isactive at any given time. In some embodiments, the illumination patternscan be pulsed in an alternating pattern (e.g. in an A, B, A, B pattern).The pulsed emissions can also be modulated to help distinguish thepulsed emission from other ambient light sources. Consequently, whenillumination pattern A is active, any area outside of illuminationpattern A should be devoid of light. However, generally a portion ofillumination pattern A reflecting off other surfaces first and incertain environments other ambient light can be detected by sensor 104reflecting off areas not being directly illuminated by illuminationpattern A. This reflected light detected in the unilluminated areas ofobject 106 can be subsequently used to identify reflected light whenillumination pattern B is active. Similarly, when illumination pattern Bis active, reflected light arriving from outside of illumination patternB can be subsequently used to identify reflected light during the nextpulse of illumination pattern A. So in general, the reflected light orindirect light (I_(INDIRECT)) detected originating from outside of theactive illumination pattern can be recorded. When the next illuminationpattern activates, the previously recorded indirect light (I_(INDIRECT))from the now active illumination pattern can be subtracted from all ofthe light (I_(TOTAL)) received from the active illumination pattern inaccordance with Eq(1) to identify the direct light.I _(DIRECT) =I _(TOTAL) −I _(INDIRECT)   Eq(1)

It should be noted that in some embodiments, any ambient lightreflecting off object 106 and back into sensor 104 can be filtered outby rejecting light not matching the modulation associated with theillumination patterns.

FIG. 2B shows exemplary illumination patterns A and B. The intensity ofillumination patterns A and B can be distributed in a sinusoidal patternas a function of vertical position. As depicted, illumination pattern Acan be 180 degrees out of phase with illumination pattern B, resultingin illumination pattern A having a maximum intensity value whenillumination pattern B is at a minimum intensity value. In this way, ifthe two illumination patterns were emitted simultaneously then asubstantially uniform light pattern would be created. Graph 206illustrates illumination pattern A while graph 208 illustratesillumination pattern B. Mathematically the intensity of the combinedpattern would cause the intensity value to have a substantially constantvalue equal to 1. More generally, the illumination intensity can bemodeled using Eq(2).L _(i)=½A ₀ (1+sin(2πfβ+ϕ_(i)   Eq(2)

In Eq(2), i indicates which illumination pattern of a total of Nillumination patterns is being calculated. A₀ is the amplitude of theillumination pattern. f is the spatial frequency of the light bars. β isthe angle of the vertical field of view of the sensor. ϕ_(i) representsthe shift in phase for the illumination pattern whose value isdetermined by Eq(3).

$\begin{matrix}{\phi_{i} = {\frac{2\;\pi}{N}i}} & {{Eq}(3)}\end{matrix}$

As can be appreciated, Eq(3) makes clear that the phase shift can be 180degrees for two patterns, 120 degrees for three patterns, 90 degrees forfour patterns, etc. In general, more illumination patterns can be usedto achieve more accurate results. Furthermore, in some embodiments, theshift in phase can also be varied in different manners

FIG. 2C shows illumination patterns C and D. The intensity profiles ofillumination patterns C and D are trapezoidal instead of sinusoidal. Byhaving rapidly rising and falling intensities, a more abrupt transitionbetween light bars of illumination patterns C and D can be achieved. Amore crisp transition can be beneficial in minimizing ambiguity whenfiltering the indirect light from the direct light, as will be describedin greater detail below.

FIG. 2D shows illumination patterns E, F and G. The intensity ofillumination patterns E, F and G are distributed vertically soillumination pattern F is 120 degrees out of phase from illuminationpattern E. In this way, successive light bars can be shifted verticallybut without being complementary in nature. Graphs 214, 216 and 218quantitatively show how respective illumination patterns E, F and G varyin accordance with vertical position. The third illumination pattern canbe generated by a third light source or by optics that can shift tocreate both the second and third patterns.

FIG. 2E shows how discrete pixels or sampling points can be distributedacross multiple illumination patterns. Close up view 220 shows threedifferent sampling points p1, p2 and p3 distributed within illuminationpatterns A and B. The indirect light at each of the sampling points canbe identified by performing a number of calculations for eachpixel/sampling point. In particular, Eq(4) can be used to sum up thelight S_(i) collected by the sensor during each sequential illuminationpattern.T=Σ _(i=1) ^(N) S _(i)   Eq(4)

Eq(5) can then be used to calculate the amount of direct light when theintensity of each illumination pattern varies sinusoidally.D=√{square root over ([Σ_(i=1) ^(N) S _(i)cos(ϕ_(i))]²+[Σ_(i=1) ^(N) S_(i)sin(ϕ_(i))]²)}−“subtracted image”  Eq(5)

Eq(5) sums up the amplitude of each component of the light received,when each of the illumination patterns is active, in order to representthe total amount of light emitted over the span of one set of theillumination patterns. In a two illumination pattern projection system,the subtracted image represents reflected light detected from withinillumination pattern A when illumination pattern B is active as well asreflected light detected from within illumination pattern B whenillumination pattern A is active. By adding the two sets of reflectedlight together, the distribution of reflected light across the wholefield of view can be determined. In general, this calculation assumesthat the reflected light stays substantially the same regardless ofwhich illumination pattern is active. Consequently, subtracting thesubtracted image from the total light identifies the direct light withinthe field of view. Eq(6) shows how indirect light (I) can be calculatedby subtracting the calculated direct light (D) from the total light (T).I−T−D−const[GL]  Eq(6)

In some embodiments, const[GL] can be subtracted from the total light.This constant can be optionally used to remove grey level bias whenidentifying the indirect light in the sensor field of view. In someembodiments, subtracting the grey level bias out can improve theaccuracy of the depth data detected by the sensor. The grey level biascan be a factory setting or a value that can be periodically calibratedto keep the depth detection system working well.

FIG. 2E also demonstrates how depth detection at position p2 can beproblematic for a system with only two illumination patterns. For p1 andp3 which are situated safely away from the boundary between illuminationpatterns, indirect light rejection can be straight forward since thereare only two illumination patterns to consider. For p1, whenillumination pattern A is active the received signal is equal to thedirect light+any reflected light. When illumination pattern B is active,the received signal at p1 is equal to zero direct light+any reflectedlight. The direct light can be calculated by taking the differencebetween the two signals. This yields just the direct light since thereflected light cancels out and the direct light during illuminationpattern B is equal to zero. For p3, the calculation works in a similarmanner, yielding just the direct light. Unfortunately, at p2, which islocated on the interface precisely between the illumination patterns,direct light from both patterns A and B will be detected at about thesame intensity. This means that taking the difference in values resultsin a zero value. Furthermore, areas near the interface will also sufferfrom some inaccuracies any time direct light from both illuminationpatterns is present in substantial amounts. Consequently, illuminationpatterns with sharp boundaries between the illumination patterns willhave fewer inaccuracies at the interfaces between the illuminationpatterns. However, direct light values for points near the interface canstill be calculated by interpolation. The direct light value for p3 canbe calculated by interpolation from direct light values for p4 and p5.Generally, p4 and p5 should be as close as possible to p2. For example,a processor can be configured to select an interpolation point p4 withan amount of direct light from illumination pattern B that falls below apredetermined threshold.

FIG. 3A shows a first optics assembly 300, which is made up of a groupof light shaping components positioned in front of a light source 302.In some embodiments, light source 302 can be an infrared laser diode.Light source 302 emits light that passes through a first light shapingcomponent, collimating lens 304. Collimating lens 304 can be configuredto focus light 306 emitted by light source 302 towards a second lightshaping component, refractive optical element 308. Refractive opticalelement 308 tilts focused light 306 by an angle θ and elongates thelight vertically to generate a super-gaussian beam 310 that is directedat a third light shaping component, diffractive optical element 312.Diffractive optical element 312 then multiplies the super-gaussian beam310. While super-gaussian beam 310 is depicted for illustrative purposesas being multiplied five times, this number can vary. For example, insome embodiments, diffractive optical element 312 can be configured tomultiply the super-gaussian beam 25 times. The number and thickness ofmultiplied super-gaussian beams 310 can be selected to match thevertical field of view of an associated imaging sensor. Whensuper-gaussian beams pass through micro-lens array 314, micro-lens array314 spreads each super-gaussian beam horizontally to create anillumination pattern that illuminates regions 316, as depicted.Micro-lens array 314 can by dual sided (as depicted), single sided orcylindrical. In some embodiments, regions 318 and regions 316 can beabout the same size. Light from a second optics assembly can beconfigured to illuminate regions 318. In some embodiments, the opticsassemblies can emit light in complementary patterns so that one ofhorizontal regions 316 and 318 is illuminated at any given time.

FIG. 3B shows a second optics assembly 320, which is made up of a groupof light shaping components positioned in front of a light source 322.In some embodiments, light source 322 can be an infrared laser diode.Light source 322 emits light that passes through a first light shapingcomponent, collimating lens 324. Collimating lens 324 can be configuredto focus light 326 emitted by light source 322 towards a second lightshaping component, refractive optical element 328. Refractive opticalelement 328 tilts focused light 326 by an angle −θ and elongates thelight vertically to generate a super-gaussian beam 330 that is directedat a third light shaping component, diffractive optical element 332. Insome embodiments, orienting supper Gaussian beam 330 in a directionopposite from the direction of super-gaussian beam 310 can reduce a riskof cross-talk between the light sources. Diffractive optical element 332then multiplies the super-gaussian beam 330. While super-gaussian beam330 is depicted for illustrative purposes as being multiplied fivetimes, this number can vary. For example, in some embodiments,diffractive optical element 312 can be configured to multiply thesuper-gaussian beam 25 times. The number and thickness of multipliedsuper-gaussian beams 330 can be selected to match the vertical field ofview of an associated imaging sensor. When super-gaussian beams passthrough micro-lens array 334, micro-lens array 334 spreads eachsuper-gaussian beam horizontally to create an illumination pattern thatilluminates regions 318, as depicted. In this way, light sources 322 and302 can cooperatively illuminate regions 316 and 318. Illumination ofregions 316 and 318 can be staggered in different patterns. For example,regions 316 and 318 can be sequentially illuminated so that light shinesin both regions for about the same amount of time.

FIG. 3C shows another optics assembly 340, which is made up of threelight shaping component positioned in front of a light source 342. Insome embodiments, light source 342 can be an infrared laser diode. Lightsource 342 emits light that passes through a first light shapingcomponent taking the form of collimating lens 344. Collimating lens 344can be configured to collimate light 346 emitted by light source 342travelling toward a second light shaping component taking the form ofoptical element 348. Optical element 348 can include both a refractivesurface 350 on a first side of optical element 348 and a diffractivesurface 352 on a second side of optical element 348. Refractive surfaces350 and diffractive surfaces 352 can take the form of polymer materialmolded onto opposing sides of a glass or polycarbonate substrate. Whencollimated light 336 passes through refractive surface 340, the light istilted by an angle θ and elongated into a super-gaussian beam 354 withinoptical element 348. When the super-gaussian beam 354 passes throughdiffractive surface 352, the super-gaussian beam 354 can be multipliedinto multiple super-gaussian beams 354. When super-gaussian beams 354pass through micro-lens array 356, micro-lens array 356 spreads eachsuper-gaussian beam 354 horizontally to create an illumination patternthat illuminates regions 316, as depicted. In this way, light source 342illuminates regions 316.

FIGS. 4A-4B show a projector assembly 400 with two light sources thatincorporates an optics assemblies for each light source similar tooptics assembly 300. FIG. 4A shows a top view of projection assembly400. Projection assembly 400 includes light sources 402 and 404. Lightsources 402 and 404 can both be mounted to rigid substrate 406. In someembodiments, rigid substrate 406 can be formed from an alumina ceramic.Rigid substrate 406 keeps light sources 402 and 404 from shiftingposition relative to one another. Rigid substrate 406 can also have alow coefficient of thermal expansion that reduces shifting of lightsources 402 and 404 with respect to the optics assemblies.

Light source 402 shines light through a first portion of dualcollimating lens 408, which focuses the light towards optics assembly410. A second portion of dual collimating lens 408 focuses light emittedby light source 404 towards optics assembly 412. In some embodiments,dual collimating lens 408 can be replaced by two separate collimatinglenses that accomplish the same function. Optics assembly 410 and 412can each include a refractive optical element similar to 308, adiffractive optical element similar to 312 and a micro-lens arraysimilar to 314 for spreading the light from each light source in anillumination pattern. Optics assembly 410 can be slightly different fromoptics assembly 412, making the illumination pattern generated by lightsource 404 vertically offset from the illumination pattern generated bylight source 402 so that the illumination patterns are complementary.This allows the light bars from one illumination pattern to bepositioned between the light bars of the other illumination pattern. Inthis way, the illumination patterns generated by light sources 402 and404 cooperate to uniformly cover a surface. In some embodiments, therefractive optical element can shift light from light source 404 in anopposite direction from the light generated by light source 402.

Projector assembly 400 can also include a processor 414 mounted on PCB416 and configured to synchronize output from light sources 402 and 404.For example, processor 414 can be mounted to PCB 416 and configured todirect light sources 402 and 404 to send out staggered pulses of light,so that neither illumination pattern is active at the same time.Processor 414 can also direct modulation of light sources 404 to helpthe depth sensor distinguish the pulses of light from other ambientlight sources. In some embodiments, processor 414 can also be incommunication with a sensor configured to receive the pulses of lightafter being reflected off objects within the sensor's field of view.

FIG. 4B shows a side view of projection assembly 400. In particular,light source 404 is shown elevated by rigid substrate 406. Rigidsubstrate can be inserted into a notch defined by PCB 416. Rigidsubstrate 406 can also form a base for projector housing 418 ofprojector assembly 400. Projector housing 418 can define a ledge 420 forsupporting dual collimating lens 408.

FIGS. 5A-5B show views of a multiple light source projector assembly 500utilizing folded optics. FIG. 5A shows how projector assembly 500includes two separate sets of optics, optics assemblies 410 and 412,which receive emitted light from folded optics 502 of collimating lens408. Folded optics 502 allows light sources 402 and 404 to be positionedcloser to collimating lens 408 by shifting light path 504 laterally,thereby allowing an overall reduction in the height of projectorassembly 500.

FIG. 5B shows how by shifting light path 504 laterally, the height ofprojector assembly 500 can be reduced, thereby allowing projectorassembly 500 to be packaged within a smaller form-factor device. Inparticular, the laterally shifted light path 504 allows a length of thelight path to be split into horizontal and vertical segments. Theoverall height of projector assembly 500 is reduced since the portion ofthe light path within the horizontal segment does not need to beincorporated within the overall height of projector assembly 500. Adirection of the light path through folded optics 502 is redirected byoptically reflective surface 506, which reorients the light from ahorizontal orientation to a vertical orientation. In some embodiments,optically reflective surface 506 can be mirrored to

FIG. 5C shows projector assembly 510, which can have a shorter overallheight than projector assembly 500. Collimating lens 408 can includeboth folding optics 502 and cylindrical lens surface 508. Cylindricallens surface 508 can partially collimate the light emitted by lightsource 404 by narrowing the width of the light entering collimating lens408. Folded optics 502 can be shorter vertically due to the narrowedbeam width of the light emitted by light source 404. The light henbecomes fully collimated upon exiting collimating lens 408. In this way,a height of collimating lens 408 can be reduced

FIGS. 6A-6B show side views of a projection assembly 600 using a singlelight source 602. FIG. 6A shows projection assembly 600 in an inactiveconfiguration. Because projection assembly 600 only includes a singlelight source 602, in order to create two different illumination patternsprojection assembly 600 includes a linearly actuated optics 606configured to generate two complementary illumination patterns. Optics606 can be linearly actuated by piezo-electric motor 608, which actuatesoptic 606 between two or more positions by rotating linkage 610 twopositions shown in FIG. 6B. Piezo-electric motor 608 can be configuredto oscillate optic 606 back and forth at a rate allowing light source602 to sequentially project complementary illumination patterns 612 and614. Light source 602 can be synchronized with the oscillation rate ofoptic 606 so that light source 602 emits light only when optic 606 is ina position corresponding to one of the complementary illuminationpatterns. It should be noted that while only two illumination patternsare shown that piezo-electric motor 608 can also be configured to definethree or more different illumination patterns.

FIG. 7 shows a diagram depicting interaction between differentcomponents of the depth detection system described above. The top of theflow chart indicates the beginning of the interaction and progresses onmoving down the flow chart. A projector of a depth detection systemsends out alternating first and second illumination patterns. Objectswithin a sensor field of view of the depth detection system reflectportions of the first and second illumination patterns back into thesensor of the depth detection system. The light travelling directly fromthe projector to the object and back (direct light) will arrive back atthe sensor before light bouncing off another surface prior to returningto the sensor (indirect light) does. Consequently, a time of flightdepth detection system will incorrectly increase the distance of anobject from the sensor when indirect light is considered. The sensorthen sends the light received from the first and second illuminationpatterns to the processor. The processor can then be configured tofilter out indirect light from the total light received so that onlylight that travels directly from the project to the object and back tothe sensor is considered when determining the distance between thesensor and the objects within the sensor field of view. The processorcan then assign the objects within the sensor field of view toappropriate depth planes of a display associated with the depthdetection sensor. Finally, the processor can send imagery to depthplanes corresponding to the various objects within the sensor field ofview.

The various aspects, embodiments, implementations or features of thedescribed embodiments can be used separately or in any combination.Various aspects of the described embodiments can be implemented bysoftware, hardware or a combination of hardware and software. Thedescribed embodiments can also be embodied as computer readable code ona computer readable medium for controlling manufacturing operations oras computer readable code on a computer readable medium for controllinga manufacturing line. The computer readable medium is any data storagedevice that can store data, which can thereafter be read by a computersystem. Examples of the computer readable medium include read-onlymemory, random-access memory, CD-ROMs, HDDs, DVDs, magnetic tape, andoptical data storage devices. The computer readable medium can also bedistributed over network-coupled computer systems so that the computerreadable code is stored and executed in a distributed fashion.

The foregoing description, for purposes of explanation, used specificnomenclature to provide a thorough understanding of the describedembodiments. However, it will be apparent to one skilled in the art thatthe specific details are not required in order to practice the describedembodiments. Thus, the foregoing descriptions of specific embodimentsare presented for purposes of illustration and description. They are notintended to be exhaustive or to limit the described embodiments to theprecise forms disclosed. It will be apparent to one of ordinary skill inthe art that many modifications and variations are possible in view ofthe above teachings.

What is claimed is:
 1. A depth detection system, comprising: aprojection system, comprising: a projector housing having a rigidsubstrate, a first light source configured to emit light through a firstplurality of light shaping components so as to produce a firstillumination pattern, the first light source being mounted to the rigidsubstrate, and a second light source configured to emit light through asecond plurality of light shaping components so as to produce a secondillumination pattern complementary to the first illumination pattern,the second light source being mounted to the rigid substrate adjacent tothe first light source, wherein the first and second illuminationpatterns are configured to sequentially pulse such that only one of thefirst and second illumination patterns is active at any given time; animaging sensor proximate the projection system and configured to receivelight emitted by the first and second light sources after beingreflected off objects within a field of view of the imaging sensor; anda processor configured to: subtract a portion of light received by theimaging sensor when the first illumination pattern is active from lightreceived by the imaging sensor when the second illumination pattern isactive; and calculate a distance between the depth detection system andthe objects within the field of view of the imaging sensor by measuringan amount of time for light emitted by the first and second lightsources to reflect off the objects within the field of view of theimaging sensor and return to the imaging sensor.
 2. The depth detectionsystem as recited in claim 1, wherein the first and second light sourcesare infrared laser diodes.
 3. The depth detection system as recited inclaim 1, wherein the imaging sensor has a global shutter.
 4. The depthdetection system as recited in claim 1, wherein the first and secondlight illumination patterns are pulsed in an alternating pattern.
 5. Thedepth detection system as recited in claim 1, wherein the firstplurality of light shaping components comprises a diffractive opticalelement and a micro-lens array.
 6. The depth detection system as recitedin claim 5, wherein the first plurality of light shaping componentsshapes light emitted by the first light source into a first plurality ofparallel light bars distributed across the field of view of the imagingsensor.
 7. The depth detection system as recited in claim 6, wherein thesecond plurality of light shaping components shapes light emitted by thesecond light source into a second plurality of parallel light barscovering gaps between the first plurality of parallel light bars.
 8. Thedepth detection system as recited in claim 1, wherein the firstplurality of light shaping components comprises a collimating lens withfolded optics.
 9. The depth detection system as recited in claim 8,wherein the second plurality of light shaping components comprises thecollimating lens with folded optics.
 10. The depth detection system asrecited in claim 9, wherein the light projected by the first and secondlight sources is reoriented by about 90 degrees by a reflective surfaceof the collimating lens.